Interactive method of presenting information in an image

ABSTRACT

An interactive method of presenting information in an image to a user, who carries a head-mounted display device that presents an image of the surroundings. The user&#39;s head position and direction are established in relation to the surroundings and a calculating device establishes, based on this, which part of the surroundings is positioned in front of the user&#39;s head. Furthermore, the calculating device establishes where in a chain of information the user is positioned and, based on this, presents information in the form of a virtual image which is mixed with the real image. A central part of the image is then defined and fields and/or objects outside this area are shown in a non-disturbing way until the user&#39;s gaze has left the central part. The user informs the calculating device that he/she has received the information and, respectively, performed the current operations by looking at the fields/objects.

This is a national stage of PCT/SE2005/000936 filed 17 Jun. 2005 andpublished in English.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an interactive method of presentinginformation in an image and, more specifically, a method allowing theuser to interact with a system that mixes real and virtual information.

2. Description of the Related Art

In the future society, the technical systems will be increasinglycomplicated and interconnected in complex structures. At the same timethe technical maintenance staff must be more generally usable than isthe case now that a large number of specialists are employed. Themaintenance staff will therefore need an intuitive technical supportsystem which is easy to use.

Such a system can be based on Mixed Reality (MR) which comprisestechniques and methods where real and virtual visual information aremixed. By approaching an object, the user of an MR system can, based onhis own needs, have relevant information about the object projected inhis field of vision. The information may concern a system or anindividual component. It may also involve a performance operation, forinstance, how a component is to be dismounted, or the tightening momentwhich is to be applied in mounting. The user can also via the MR systemobtain support in fault localisation and have a corrective measurepresented to himself.

The technical development has made it possible, and makes it possible toan increasing extent, for an individual to register with his sensesvirtually created information which to some extent corresponds to theinformation he is used to experience from reality. This means that it ispossible to divide reality into two parts, the real world and thevirtual world. The two worlds can be mixed by means of technical devicesso that a user of such a technical device can perceive the two worlds asone.

In MR, the virtual and real worlds can thus be mixed. In a first step,it is possible to start from only mixing visual image information fromthe two worlds, but in a subsequent step, also acoustic andtactile/haptic information from the two worlds can be included. For thepurpose of facilitating the understanding of the invention describedbelow, the description is restricted mainly to comprise merely visualimage information.

MR can be used in many different applications, such as for training,repair, maintenance and manufacture, in various fields, for instance inmedicine, manufacturing industry, adventure industry and in militarycontexts.

SUMMARY OF THE INVENTION

The present invention provides a solution to the problem of presentingin each moment relevant information to a user, while taking the user'seducation level and previous experience into consideration, by designingthe information in the manner described herein. Particularly, aninteractive method of presenting information in an image to a user isdescribed in which the user carries a head-mounted display device whichpresents an image of the surroundings to the user. The user's headposition and direction are established in relation to the surroundings,and a calculating device establishes, based on the preceding step, whichpart of the surroundings is positioned in front of the user's head. Thecalculating device also establishes where in a hierarchical scheme theuser is positioned as regards information about and performed operationsconcerning that part of the surroundings which is positioned in front ofthe user's head. Further, the calculating device establishes, based onthe preceding step, which information is to be presented to the user andpresents it in the form of a virtual image which is mixed with the realimage. A central part of the image is defined, with there being in thearea outside the central part fields and/or objects that are adapted tobe included in a transfer of messages from the user to the calculatingdevice. The user's direction of gaze at the image is established and itis made clear when the point of fixation in the image is in a field oran object. Finally, the calculating device is informed that the user hasreceived the information and, respectively, performed the currentoperations depending on how the point of fixation is positioned infields and/or objects.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described in more detail in the following withreference to the accompanying drawings, in which

FIG. 1 shows direct vision and mixing of the reality and virtualinformation using what is called optic-see-through,

FIG. 2 shows indirect vision via a camera and a display device andmixing of the reality and virtual information using what is calledvideo-see-through,

FIG. 3 shows six degrees of freedom for a user's head,

FIG. 4 shows an example of a marker with a unique pattern,

FIG. 5 shows a mixed image with a marker, a virtually created object anda real object and fields/objects inserted in the image, in which thepoint of fixation of the user's eyes can be positioned to indicate afunction, and also a cross which indicates the point of fixation of theuser's eyes in the image,

FIG. 6 shows an example of a head-mounted system for MR with anintegrated gaze tracker,

FIG. 7 is a general view of a concrete implementation of the invention,

FIG. 8 shows the different parts that are included in the head-mountedequipment in FIG. 7, and

FIG. 9 is a flow chart showing how computer and software can operate ina system according to the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Further scope of applicability of the present invention will becomeapparent from the detailed description given hereinafter. However, itshould be understood that the detailed description and specificexamples, while indicating preferred embodiments of the invention, aregiven by way of illustration only, since various changes andmodifications within the spirit and scope of the invention will becomeapparent to those skilled in the art from this detailed description.

For the purpose of facilitating the understanding of the content of theinvention, the further description is based on a concrete example whichis called “live manual” and which describes a user who carries atechnical device, in which the visual virtual and real worlds can bemixed and presented to him. His task is assumed to involve dismountingan engine block based on visual instructions. The technical steps ofsuch a process will be described below.

Direct and Indirect Vision

Real visual information is the information that originates from the realworld. Monitoring can be made directly, i.e. the visual radiation fromthe surroundings 3 reaches the user's eye 1 directly or via some kind oftransmissive or partly transmissive optical device 2, for instance via apair of goggles, see FIG. 1. Monitoring can also occur indirectly, forinstance via a camera 4 and a display device 5, see FIG. 2. The camerahere monitors the visual radiation from the surroundings, converts theradiation into an electric signal which is transferred to the displaydevice where the signal is converted into an image that describes thesurroundings. The camera 4 and the display device 5 can be designed insuch a manner that they can be placed close to the user's head and eyes.In this case the camera follows the movements of the user's head, thusresulting in head-controlled indirect vision.

A camera used in such a context does not have to be sensitive to visualradiation only, but can be sensitive in other ranges as well, forinstance in the near IR range, the thermal IR ranges or the UV range.The camera can also detect the surroundings by multispectral bands. Byletting the camera be sensitive to radiation both inside and outside thereception area of the human eye, the indirect vision may help a user toperceive the surroundings in a more comprehensive manner than in thecase using direct vision.

Since the camera monitors the surroundings and converts what it seesinto an electric signal, it is possible to let this signal pass acomputer which improves, changes or converts the image represented bythe electric signal, for instance by noise reduction, contrastamplification or amplification of small local signal variations in shadyor light areas.

Creating Virtual Visual Information

Virtual visual information is here information which is created by meansof a computer or some other image-creating device. The virtual visualinformation is presented to the user via a display device 5. The virtualvisual information can originate, for instance, from a database or becalculated by the computer. Examples of virtual visual information inthis context are two-dimensional images (with image and/or textinformation), two-dimensional objects (for instance an arrow),three-dimensional text, three-dimensional models (objects), videosequences, particle systems (for instance flickering smoke/fire) andmoving simulated animated sequences.

Mixing Virtual and Real Visual Image Information

The virtual visual information, described above, can be mixed with thereal visual information using mainly two different techniques,optic-see-through and video-see-through.

In the first technique, optic-see-through, see FIG. 1, the user sees thereal visual world via a transmissive or partly transmissive opticaldevice 2, into which the virtual visual information is reflected. Thevisual radiation which describes virtual image information is generatedby a computer 7 based on line-up data from a line-up device 8. Thevirtual information is presented via a display device 5 and reflectedinto the user's eye via a beam splitter 6.

In the second technique, video-see-through, see FIG. 2, the user seesthe real visual world via a camera 4 and a display device 5. The imagefrom the camera passes a computer 7 or some other image-calculating unitbefore being presented to the user via the display device. The computercan then electronically or in a computerised manner intermix virtualvisual information generated in the same way as in the preceding examplebased on line-up data from a line-up device 8. This intermixing can bemade at different degrees, all the way from the virtual visualinformation being slightly superimposed over the real up to the virtualvisual information replacing relevant real visual information.

Line-Up Process

For mixing of the virtual and real visual information to take place in acorrect manner, some kind of line-up process must be used. The virtualvisual information is in most cases related to a position in a scene inthe real world; for instance virtual visual information describing howelectric wires are placed in a wall is related to the position of thereal wall, since otherwise the wires—virtually described—cannot becorrectly superimposed on the wall, seen with the user's eyes.

Line-Up in Optic-See-Through

Starting from the user carrying head-mounted equipment foroptic-see-through, the computer must somehow know in which direction theuser has turned his head to be able to calculate which virtual visualinformation is to be presented and also how and where in the image inthe display device this should occur. In an advanced form, also theposition of the eye in relation to the position of the equipment isused, since the eye is the human image detector that records the mixedvisual information. A minor displacement of the equipment on the user'shead may result in the real and virtual visual information beingdisplaced in an undesirable way.

There are many ways of lining up the user's position and direction inrelation to a scene in the real world. Various techniques have beendeveloped and line-up equipment, trackers, can be grouped according towhich physical principle is used: acoustic, magnetic, based on the lawof inertia, optical and mechanical. It is also possible to combine twotrackers that are based on different principles and, thus, obtain ahybrid tracker.

A user's relationship to his surroundings can be lined up by detectingboth position, x, y and z, and direction, i.e. angles of rotation head,pitch and roll, see FIG. 3. In the figure head is designated h, pitch pand roll r. If position and direction in all directions are lined up, aline-up in six degrees of freedom has been made. In some cases, forinstance when the user is sitting more or less strapped in a chair, itmay be sufficient to measure only head, pitch and roll, i.e. only threedegrees of freedom.

If a camera is integrated with the head-mounted equipment and thedirection of the camera is aligned with the optical direction of theequipment, the camera image can be read by a computer which by means ofsoftware can detect patterns or local properties in the image and inthis manner orient itself in the surroundings.

Line-Up in Video-See-Through

Starting from the user carrying head-mounted equipment forvideo-see-through, the information of the camera image can be used bythe computer in the same way as described above regardingoptic-see-through. The computer can thus detect local properties orpatterns in the image from the camera and, based on this information,orient itself. A local property can be, for instance, the corner of atable or a nut. However, it may be difficult for the computer to findsuch local properties in the image from the camera since they can beviewed, for instance, from different angles of vision and in differentlighting situations.

It is therefore suitable to assist the computer in orienting itself inthe image by arranging patterns which have been specially designed toallow the computer to find them in the image from the camera. Such aspecially designed pattern, hereinafter called a marker, should bedesigned so that the pattern of the marker is unique and the positionand direction of the marker, i.e. its six degrees of freedom, can bedetermined from the marker. A marker, 9 in FIGS. 4 and 5, can simply bea black-and-white unique pattern 10, surrounded by a black frame 11against a white background 12. A marker may also consist of a number oflight-reflecting dots, a number of light-generating dots, for instancelight-emitting diodes, or just be an ordinary image depicting something.

A marker can, in addition to assisting the computer in orienting itself,be connected to some kind of virtual visual information, or in a moregeneral application to some other virtual information, for instanceacoustic. Information about how and where the virtual visual informationis to be presented to the user can also be connected to each marker. If,as shown in FIG. 5, a marker 9 is placed on an engine block 13 and inthe vicinity of a bolt 14, the computer can, for instance, correctlyposition a virtually created spanner 15 to mark that the bolt is to beunscrewed by means of a spanner.

The computer can orient itself in the surroundings by detecting a markerin the camera image which is also seen by the user. As an alternative,the field of vision of the camera can be larger than the field of visionpresented to the user via the display device. In such a case, thecomputer can find markers which are positioned wholly or partly outsidethe image presented to the user. In a further development, one or morecameras, which are arranged in a fixed and known relationship to thecamera that gives the user his image, can be used to detect markers in acompletely different direction from the direction in which the userdirects his head. Owing to the known relationship between the cameras,relevant virtual visual image information can still be positioned in acorrect manner in the image which the user sees.

Markers and Order of Execution

By using a number of different markers which are placed in differentpositions, for instance on an engine block, a user carrying ahead-mounted video-see-through system connected to a computer can obtaina visual description how he should proceed, for instance, to dismountthe engine. The user obtains a live manual.

Each marker is connected to some kind of virtual visual information andas the user approaches the current object, the engine block, of courseall markers cannot provide virtual visual information simultaneously,but there must be a specific order regarding how the virtual visualinformation is to be displayed, i.e. how the various operations are tobe performed, for example, in order to dismount the engine.

The invention is established in different chains of information, and thevarious manipulations that are to be performed by the user are arrangedin chains of execution. It is a structure system where information at acertain level in a hierarchical system normally leads to newinformation. The user should also by messages to the calculating devicebe able to jump back and forth in the hierarchical system to obtaininformation he considers to be necessary.

Moreover the system can in a more advanced design adapt the informationto the user's level of knowledge and experience. It is conceivable thatthe system presents “all” information on a first occasion to a certainuser. If the user then is to repetitively perform the same manipulationson a sequence of objects, such as engines, the system can restrict theinformation step by step according to a predetermined plan of how theuser's experience increases. However, as mentioned above the user can atany time request more information.

If besides a system is used which requires the user to identify himself,the system can also take into consideration what the user “should beable to perform” in consequence of what the user has performed onprevious occasions. Since a system according to the invention mustcomprise an gaze tracker, see further below, to determine the point offixation of an eye in the image, it would be possible to combine thiswith an opportunity to identify the user by eye recognition.

In the normal case, a marker is considered when determining whichinformation is to be presented first when the entire marker is visiblein the image. A more advanced system contains, however, severalvariants. A marker can be active/passive, executable/non-executable,acknowledged/unacknowledged etc.

The basic principle implies that a marker is active when positioned inthe camera image, which usually is also the image presented via ahead-mounted display device, and passive when positioned outside theimage. As a rule, when a marker is active, the virtual informationconnected to the marker is shown whereas, when a marker is passive, noinformation at all is shown.

In a more developed system, a number of conditions can be introduced fora marker to be active, for instance it must be positioned within acertain distance to the user, it must have the correct priority, certainoperations must be performed previously etc. In such a system, a passivemarker can also, depending on the situation, show virtual information,such as “Do not perform this operation at the moment”.

For the system to be able to correctly conduct a user, for instance, ina mounting sequence, certain operations must occur in the correct order.Therefore it has been found relevant to introduce the termacknowledgement of marker or, more correctly, acknowledgement of theinformation which is connected to the marker, so that the system, forexample, can obtain information that a user has performed a certainoperation/manipulation. After a user has performed a certain operation,he/she therefore acknowledges that he/she has been informed about orperformed an operation associated with the information which isconnected to the marker. This acknowledgement can occur by means of acommand made, for instance, by the user's hand, voice or eye. For theuser to be aware of the state of acknowledgement, a symbol can beconnected to the marker which shows the status thereof. For example, itcan be a symbol in the form of a hand where the colour thereof marks thecurrent state of acknowledgement.

Interaction in Operations of Execution

The user carrying the head-mounted equipment and receiving instructionsabout how he/she should perform the various operations, for instancedismount the engine, has in most cases both hands occupied by the actualdismounting and besides his/her hands can be dirty. In this situation,the user must, as discussed above, be able to acknowledge somehow thevarious operations. Here the condition can be applied that the useralready carries head-mounted equipment and supplement this with a gazetracker (for one or both eyes), for instance as shown in FIG. 6. In thefigure, the eye is designated 1, part of the surroundings 3, a camera 4,a display device 5, a computer 7, a camera 16 sensitive to near IRradiation, a filter 17 letting through only near IR radiation, near IRlight sources 18, a spherical mirror 19, a hot mirror 20, i.e. anoptical device having the function that near IR radiation is reflectedas in a mirror while visual radiation is transmitted, and a beamsplitter 21. If the direction of both eyes is detected, this can be usedto create a correct stereographic virtual presentation, provided that acamera will be used for each eye, so that also the camera image isstereographic.

The gaze tracker, or the gaze trackers, detects where in the image theuser has his point of fixation. A number of fields/objects are marked inthe image from the camera—i.e. in the image of the surroundings, seeFIG. 5. A field in the image is designated 22, a three-dimensionalobject in the image 23, and a cross 24, which describes where in theimage the point of fixation of the eye is positioned.

The fields/objects, which suitably can be positioned close to thecorners in the image and can be shaped in various ways, for instance asthe arrow 22 in FIG. 5, for the function of the field/object to beunderstood immediately. The field/object can be shaped as a fullytransparent surface in the image, surrounded by a coloured frame wherethe colour of the frame designates the current function. The functioncan also be indicated as an amount of text in the field/object. Thefield/object can also be more or less transparent or be completelyfilled with a colour, a pattern or an image. The field/object can alsobe three-dimensional, stationary or moving. When the user looks at oneof the fields/objects, this can be indicated by the shape, colour,movement or contents of the field/object being changed somehow, so thatthe user understands that the system knows that the user looks at thefield/object.

By interacting with a field/object, the user can acknowledge that he/shehas received information, performed an action or requests stepwisemovement in a hierarchical scheme of a sequence of execution comprisedby the system. The actual command, the acknowledgement, can be given tothe system in various ways. A simple way is to keep looking at a specialfield or object for a sufficient length of time. It is also possible tolet the eye's passing of a number of fields/objects in a predeterminedsequence give the current command. One way may involve letting the eyerapidly move to the periphery of the image and after that quickly to thefield in which the user is interested. Another option is to let theposition of the eye in a field or an object be combined with acompletely different action, for instance a voice command or a commandperformed by some other part of the body than the eye, for instance theuser's foot.

A field/object can also be positioned in different virtual focal planesif a camera and a display device are used for each eye. A field/objectcan be experienced to be close to the user or be experienced to bepositioned far away owing to the stereo effect. It is here possible tolet the various fields/objects take different distances, experienced bythe user, to the user for the purpose of giving the user some kind offeedback.

When looking around, the user uses his head for large turnings and hiseye for minor turnings. When a user uses gaze-controlled MR equipmentand looks around, the point of fixation of his eye normally moves withina limited area around the centre of the image (his head makes the largemovements). This can be applied so as not to have to show fields/objectsbefore they are needed. It can be disturbing if there are fields/objectsin the image when the user is not interested in being able toacknowledge or give a command, even if they are positioned at the edgeof the image. In this case the user defines a central part of the imageand not until the point of fixation of his eye leaves this centralimage, the fields/objects are shown.

It is still more disturbing if the point of fixation is always shown inthe image. The invention therefore does not show the point of fixationin the image when it is positioned in the central part of the image, orshows it in a non-disturbing manner, for instance as a dot. Outside thecentral part, it is normally no problem to show the point of fixation,and it is important to know where one looks when one wants to interactwith the fields/objects. With reference to FIG. 5 and designations inthe same, one way can be to mark when the point of fixation 24 in theimage is in a field 22 or an object 23 by the field/object changing itsappearance, for instance colour or shape. Another way is to alwayspresent the point of fixation in the image in a distinct manner, forinstance as a cross 24, when the point of fixation is outside thecentral part of the image.

It may also be important to know when the calculating device 7 hasperceived a delivered message. This can occur by the currentfield/object 22, 23 changing its appearance, for instance colour orshape, as this occurs. It would also be possible to receive thisinformation only by the image or a part thereof changing; for instancehe next step in the chain of information or execution can be presented.

As an example, a concrete implementation of the invention will bepresented in the following. FIG. 7 shows how a marker 9 with awell-defined pattern is placed on a physical object 25. A user 26 hasturned his head towards the physical object and the marker. On his head,the user has equipment 27, by means of which he can see both the realimage information from the physical object and also superimposed virtualimage information which is connected to the position and direction inrelation to the marker.

With the head-mounted equipment 27, the user's eye can also be used forinteraction between the user and a computer 7, in which software forgaze-controlled MR is implemented. The head-mounted equipment isconnected to the computer by means of a wire 28 which contains severaltransmission channels and a current supply channel.

FIG. 8, where the eye is designated 1, shows different parts included inthe head-mounted equipment:

A camera system comprising:

-   -   Colour camera 29 with, for instance, FireWire interface    -   Lens 30 for colour camera 29    -   Mirror 31 for colour camera 29 and lens 30. The function of the        mirror is to reduce parallax between the camera and a        microdisplay system which will be presented below.

A microdisplay system comprising:

-   -   Microdisplay 32 of the transmissive type    -   Illumination 33 for microdisplay 32    -   Drive electronics 34 for microdisplay 32    -   Beam splitter 35    -   Spherical mirror 36

A gaze tracker system comprising:

-   -   Black-and-white camera 37 with good sensitivity in the near IR        range. The camera can have, for example, FireWire interface.    -   Near IR filter 38 filtering off visual radiation    -   Lens 39 for camera 37    -   Near IR illumination 40 which on the one hand illuminates the        eye 1 and, on the other, causes reflections on the cornea of the        eye.    -   Hot mirror 41, i.e. mirror reflecting near IR radiation and        transmitting visual radiation at an angle of incidence of approx        45 degrees

FIG. 9 shows in the form of a flow chart how a computer and implementedsoftware can operate in a system for gaze-controlled MR.

-   -   P1. Image information describing the reality, 9 and 25 in FIG.        7, and monitored by a camera, 29 in FIG. 8.    -   P2. The recorded image is transferred via wire 28 and FireWire        interface to FireWire interface in the computer, 7 in FIG. 7.    -   P3. This image information then proceeds to marker detection        which in turn consists of    -   P4. thresholding of image information and    -   P5. determining which marker is involved and the six degrees of        freedom thereof.    -   P6. Information about which marker is sent to a database where        virtual image information connected to markers is stored and    -   P7. the six degrees of freedom of the marker are sent to the        computer graphics system where the image information from the        camera (P1) is mixed with the virtual image information from the        database (P6). How this mixing occurs is determined on the one        hand by the six degrees of freedom of the marker from (P5) and,        on the other hand, by information from the database (P6).    -   P8. The composite image information from the computer graphics        system (P7) proceeds to the head-mounted equipment, 27 in FIG.        7, which contains, inter alia, a microdisplay system.    -   P9. At the same time the eye is monitored by a camera, 37 in        FIG. 8.    -   P10. The monitored image is transferred via wire, 28 in FIG. 7,        and FireWire interface to FireWire interface in the computer, 7        in FIG. 7.    -   P11. This image information then proceeds to detection of pupil        and reflection which in turn consists of    -   P12. filtering of image information,    -   P13. thresholding of image information and    -   P14. determining the position of the pupil and the positions of        the reflections on the cornea caused by near IR illumination, 40        in FIG. 8.    -   P15. These position data are introduced into a system of        equations from which the direction of the user's eye falls out.        Eye direction data proceed to the computer graphics system (P7)        where the point of fixation is marked, for instance as a cross        in the image.

The invention being thus described, it will be apparent that the samemay be varied in many ways. Such variations are not to be regarded as adeparture from the spirit and scope of the invention, and all suchmodifications as would be recognized by one skilled in the art areintended to be included within the scope of the following claims.

1. An interactive method of presenting information in an image to a userwithin surroundings, using a system in which the user carries ahead-mounted display device which presents an image of the surroundingsto the user, said system operating in accordance with at least one chainof information having associated manipulations arranged in at least onechain of execution that is performed by a user in a hierarchicalsequence of steps to complete a task, said method comprising the stepsof: establishing the user's head position and direction in relation tothe surroundings; establishing, by a calculating device of said systembased on the user's head position and direction, which part of thesurroundings is positioned in front of the user's head as well as theuser's direction of gaze; establishing, by the calculating device, wherethe user is in a chain of information about the part of the surroundingsthat is positioned in front of the user's head, and where the user is ina chain of executed operations as regards the same part of thesurroundings; establishing, by the calculating device, on the basis ofwhere the user is in the chain of information and where the user is inthe chain of executed operations associated with said part of thesurroundings, which information in the hierarchical sequence is next tobe presented to the user and presenting the information in the form of avirtual image which is mixed with a real image as a mixed image thatenables the user to perform a current operation as a next step withinthe hierarchical sequence for completing the task; defining a centralpart of the mixed image, an area outside the central part includingfields and/or objects that are shown in a way that is not disturbing tothe user until the user's direction of gaze has left said central partof the image and that are adapted to be included in a transfer ofmessages from the user to the calculating device, said transfer ofmessages enabling the user to move back and forth in the hierarchicalsequence to obtain information needed to perform the current operation;and the calculating device being informed that the user has received theinformation and, respectively, performed the current operation based onhow the user's direction of gaze is positioned in one or more of thefields and/or objects such that the user, by changing the user'sdirection of gaze, initiates said transfer of messages to thecalculating device; recognizing, by said calculating device, saidtransfer of messages and establishing, in response to said transfer ofmessages, which information in said chain of information is next to bepresented to the user to enable the user to perform a next step withinthe hierarchical sequence; and presenting, by the calculating device,information relating to the next step in said hierarchical sequencemaking up said chain of executed operations for performance by said userto complete said task.
 2. The method as claimed in claim 1, furthercomprising marking when the user's direction of gaze in the mixed imageis in a field or an object by said field or said object changing itsappearance.
 3. The method as claimed in claim 1, wherein the user'sdirection of gaze is presented in a distinct manner when the user'sdirection of gaze is positioned outside the central part of the image,said direction of gaze being either not presented or presented in anon-disturbing manner when said direction of gaze is in the central partof the image.
 4. The method as claimed in claim 1, further comprisingperceiving by the calculating device when the user has delivered amessage by changing the user's direction of gaze to a field or an objectoutside said central part, said field or said object serving as an inputmechanism by which the user sends said message, said calculating devicechanging an appearance of said field or said object outside said centralpart to acknowledge that the message was received.
 5. The method asclaimed in claim 1, wherein the head-mounted display device mixes thereal image and the virtual image using an “optic-see-through” technique.6. The method as claimed in claim 1, wherein the head-mounted displaydevice mixes the real image and the virtual image using a“video-see-through” technique.
 7. The method as claimed in claim 2,wherein said change of appearance of said field or said object includesa change in color or shape.
 8. The method as claimed in claim 3, whereinthe user's direction of gaze is presented as a cross when a point offixation is outside said central part of the image.
 9. The method asclaimed in claim 4, wherein said change of appearance of said field orsaid object includes a change in color or shape.
 10. An interactivemethod of presenting information in an image to a user who is positionedwithin surroundings, using a system in which the user carries ahead-mounted display device which presents an image of the surroundingsto the user, said system operating in accordance with at least one chainof information having associated manipulations arranged in at least onechain of execution that is performed by a user in a predefinedhierarchical sequence of steps to complete a task, said methodcomprising the steps of: establishing the user's head position anddirection in relation to the surroundings; establishing, by acalculating device of said system, a part of the surroundings that ispositioned in front of the user's head and the user's direction of gaze;establishing, by the calculating device with respect to the part of thesurroundings positioned in front of the user's head, where the user isin a chain of information about said part of the surroundings and wherethe user is in a chain of operations to be executed that relate to saidpart of the surroundings; determining, by the calculating device, whichinformation in the chain of information is next to be presented to theuser based on where the user is in the chain of information and wherethe user is in the chain of operations to be executed; presenting theinformation to the user in the form of a virtual image which is mixedwith a real image as a mixed image that enables the user to perform acurrent operation as a next step within the hierarchical sequence ofstep to complete the task; defining a central part of the mixed imageaccording to the user's direction of gaze, an area outside the centralpart including fields and/or objects that are not shown in a way thatwould be disturbing to the user until the user's direction of gaze hasleft said central part of the image; and said calculating device beinginformed that the user has received the information and/or performed theoperation from said chain of operations to be executed by said usersending an acknowledgment signal that is recognized by said calculatingdevice.
 11. The method as claimed in claim 10, wherein said user sendssaid acknowledgment signal by looking at a field or an object outsidesaid central part, said field or said object serving as an inputmechanism for the user to send said acknowledgment signal, and furthercomprising marking, by said system, when the calculating device hasperceived a delivered acknowledgment signal by the field or the objectoutside said central part changing its appearance.
 12. The method asclaimed in claim 11, wherein said change of appearance of said field orsaid object includes a change in color or shape.
 13. The method asclaimed in claim 10, further comprising marking, by the system, when theuser's direction of gaze in the mixed image is in a field or an objectby said field or said object changing its appearance.
 14. The method asclaimed in claim 13, wherein said change of appearance of said field orsaid object includes a change in color or shape.
 15. The method asclaimed in claim 10, wherein the user's direction of gaze is presentedin a distinct manner when the user's direction of gaze is positionedoutside the central part of the image, said direction of gaze beingeither not presented or presented in a non-disturbing manner when saiddirection of gaze is in the central part of the image.
 16. The method asclaimed in claim 10, wherein the head-mounted display device mixes thereal image and the virtual image using an “optic-see-through” technique.17. The method as claimed in claim 10, wherein the head-mounted displaydevice mixes the real image and the virtual image using a“video-see-through” technique.
 18. The method as claimed in claim 10,wherein said sending of the acknowledgment signal includes movement ofthe user's direction of gaze relative to one or more fields and/orobjects in a specific pattern or by gazing at a particular field orobject for a specified time.
 19. The method as claimed in claim 10,further comprising, after said calculating device has perceived adelivered acknowledgment signal, the step of said calculating devicepresenting a next step in the chain of information or chain ofoperations to be executed in response to said acknowledgment signal.